Convergence of the Randomized Block Gauss-Seidel Method

Convergence and Comparison Theorems of the Modified Gauss-Seidel Method

In this paper, the modified Gauss-Seidel method with the new preconditioner for solving the linear system Ax = b, where A is a nonsingular M -matrix with unit diagonal, is considered. The convergence property and the comparison theorems of the proposed method are established. Two examples are given to show the efficiency and effectiveness of the modified Gauss-Seidel method with the presented n...

On the convergence of the block nonlinear Gauss-Seidel method under convex constraints

Paving the Randomized Gauss-Seidel

The Randomized Gauss-Seidel Method (RGS) is an iterative algorithm that solves overdetermined systems of linear equations Ax = b. This paper studies an update on the RGS method, the Randomized Block Gauss-Seidel Method(RBGS). At each step, the algorithm greedily minimizes the objective function L(x) = kAx bk2 with respect to a subset of coordinates. This paper describes a Randomized Block Gauss...

Breaking Locality Accelerates Block Gauss-Seidel

Recent work by Nesterov and Stich (2016) showed that momentum can be used to accelerate the rate of convergence for block GaussSeidel in the setting where a fixed partitioning of the coordinates is chosen ahead of time. We show that this setting is too restrictive, constructing instances where breaking locality by running non-accelerated Gauss-Seidel with randomly sampled coordinates substantia...

Convergence Properties of the Randomized Extended Gauss-Seidel and Kaczmarz Methods

The Kaczmarz and Gauss-Seidel methods both solve a linear system Xβ = y by iteratively refining the solution estimate. Recent interest in these methods has been sparked by a proof of Strohmer and Vershynin which shows the randomized Kaczmarz method converges linearly in expectation to the solution. Lewis and Leventhal then proved a similar result for the randomized Gauss-Seidel algorithm. Howev...